Clinical Trials

The efficacy of current antiretroviral interventions for prevention of vertical HIV transmission is both a triumph and a challenge for future efficacy studies of novel agents, including long-acting antiretroviral agents. Given the effectiveness of current antiretroviral interventions, standard randomized controlled trials to assess novel interventions targeted towards further reducing vertical HIV transmission may be infeasible due to required sample sizes. To address this study design challenge, we propose a novel augmented clinical trial design, where a randomized trial with an active comparator is augmented with a concurrent observational cohort to supplement the standard-of-care arm. The proposed design and analysis can overcome sampling biases, and an associated diagnostic can evaluate potential biases from unmeasured covariates. The proposed trial design is illustrated with an example and studied further using a simulation study. In the illustrative example, an augmented trial with a 3:1 allocation ratio in the trial component yielded an unbiased treatment effect estimate, as compared to the trial component alone, and the standard error was reduced substantially. Although the total sample size of the augmented trial was larger than a similarly powered 1:1 standard trial, the sample size of the augmented trial's trial component was half the size of the standard trial and within the range of historically similar trials, thereby providing both feasibility and potential cost advantages. Although limitations with this design remain, many of these limitations can be mitigated by careful study planning and coordination. Augmented trials provide a potentially applicable solution to studying the efficacy of long-acting antiretrovirals for infant postnatal prophylaxis against HIV.

TrialCalibre1 is an agentic system that connects clinical evidence, study design, and executable analyses. It supports reference-trial replication, direct target-trial emulation, and BenchExCal, which combines benchmarking, extension, and cal ibration sensitivity analysis. We evaluated an ASCET-based benchmark and an extension comparing clopidogrel with aspirin among adults aged 81-90 years us ing synthetic PCORnet-style claims. Two reviewers assessed benchmark extrac tion, protocol decisions, and implementation. After discussion, 53/62 and 50/62 assessable SQL checks were rated met; both reviewers rated 12/13 assessable R checks met. Protocol and proxy disagreements remained. A separate ten-question routing development set yielded 10/10 route matches and 7/7 applicable reference trial matches. The agent benchmark and extension included 6,069 and 1,602 pa tients. The extension risk ratio was 0.848 (95% CI 0.569-1.184), with a calibrated sensitivity estimate of 1.274 (0.439-3.700). The human implementation provided descriptive comparisons, not same-data validation. Dose proxies, unpopulated co variates, residual imbalance, implementation deviations, and ignored cross-cohort covariance limited interpretation. This case demonstrates a connected workflow whose protocols, code, and results can be reviewed.

BACKGROUND: Federal policy now requires sponsors to submit diversity action plans for clinical trials, yet the modifiable features of trial design associated with enrolled composition remain poorly characterized. English-language proficiency requirements are written into eligibility criteria and recorded as free text in a public registry, making them visible and, in principle, amenable to revision, but their relationship to enrolled composition has been examined only within single disease areas and in small samples of trials. We examined whether these requirements and documented language accommodation were associated with the racial and ethnic composition of enrolled participants. METHODS AND FINDINGS: We conducted a cross-sectional study of interventional trials registered on ClinicalTrials.gov with at least one US site and posted results, using a registry snapshot retrieved in October 2026. The primary cohort comprised the 44,487 trials with exclusively US sites, of which 21,924 (49.3%) reported baseline race and 17,434 (39.2%) reported baseline ethnicity with counts that reconciled with the posted denominator. Both exposures were classified from free-text eligibility criteria using regular expressions validated against independent physician review of 150 trials. For the English-language requirement classifier, sensitivity was 88.2% and specificity was 99.2%. The outcome was the trial-level proportion of enrolled participants in each category, analyzed using generalized linear models adjusted for intervention class, therapeutic area, sponsor, phase, enrollment, sites, start year, US Census region, and the population composition of the trial's site states. An English-language proficiency requirement appeared in 5,736 trials (12.9%); its prevalence rose from 7.4% of trials starting in 2010 to 26.2% in 2023 (odds ratio [OR] per decade 2.91, P<0.001), whereas documented accommodation, an eligibility statement allowing participation in a language other than English (Spanish in 99.9% of flagged trials), remained rare (788 trials, 1.8%). The requirement was associated with lower enrolled proportions of Asian (OR=0.85, 95% CI 0.77-0.93, P=0.001) and Hispanic/Latinx participants (OR=0.85, 95% CI 0.79-0.91, P<0.001), with no evidence of an association for White participants (OR=0.99, 95% CI 0.95-1.04, P=0.75). Documented accommodation was associated with a higher proportion of Hispanic/Latinx participants (OR=3.23, 95% CI 2.82-3.71, P<0.001). Geographic adjustment did not change the statistical significance of the requirement associations and attenuated the accommodation association. The main limitations were the cross-sectional design, which precludes causal inference; the restriction of enrollment analyses to trials reporting baseline demographics, which were more recent than those without such reporting; and possible misclassification of automatically classified exposures. CONCLUSIONS: English-language proficiency requirements became more than three times as common between 2010 and 2023 and were associated with lower enrolled proportions of Asian and Hispanic/Latinx participants. Less common documented language accommodation was associated with a higher enrolled proportion of Hispanic/Latinx participants. Because these criteria are investigator-written and publicly recorded, they may be a tractable target for improving enrollment equity, although whether revising them changes who enrolls requires prospective evaluation

Cash incentives are widely used in clinical research to support recruitment and retention, yet their ethical implications, particularly in high-risk settings such as HIV cure-related studies, remain inadequately understood. This study examined how key stakeholders perceive participant payment relative to other study characteristics when deciding whether to participate in HIV cure research. Using a choice-based conjoint (CBC) experiment, we assessed preferences among people living with HIV (PLWH), HIV researchers, and Institutional Review Board (IRB) members/bioethicists. Participants (N=88) completed eight choice tasks involving hypothetical HIV cure study scenarios varying across six attributes: incentive amount, invasiveness, requirement to pause HIV medications, time commitment, first-in-human status, and cost reimbursement. Across all participants, payment amount emerged as the most influential attribute (29.3%), followed by procedure invasiveness (22.3%) and the requirement to pause HIV medication (17.6%). The highest hypothetical payment level ($200 per visit) was significantly preferred. However, subgroup analyses revealed important differences: PLWH prioritized payment most strongly, whereas researchers and IRB members/bioethicists placed greater emphasis on minimizing invasiveness. The requirement to pause HIV medication was a salient concern across groups, though relatively more influential for IRB members/bioethicists than for PLWH. Time commitment and ancillary cost reimbursement had comparatively lower impact on decision-making. These findings highlight the central role of financial incentives in shaping willingness to participate in HIV cure research, raising ethical considerations regarding potential undue influence, especially in higher-risk studies. At the same time, stakeholders value minimizing procedural risks and disruptions to treatment. This study underscores the need for clearer guidance on appropriate payment practices and for balancing incentives with ethical safeguards, particularly in trials involving antiretroviral therapy interruption.

Biomedical Machine Learning (ML) is increasingly evaluated through Independent and Identically Distributed (IID) benchmarks, even when deployment involves irregular observation, missingness, distribution shift, and high failure costs. This position paper argues for a stability-oriented reporting standard for safety-relevant biomedical ML. The central requirement is not a specific model class or optimizer, but auditable evidence: performance should be reported under clinically plausible perturbations such as thinning, timestamp jitter, bursty missingness, channel dropout, and Signal-to-Noise Ratio (SNR) shift. When authors invoke training-stability diagnostics, those diagnostics should be hazard-linked: their assumptions, operational meaning, and relationship to stress-test degradation should be stated explicitly. We use controlled mechanism probes to illustrate two hazards that IID scores can hide: observation-process dependence under irregular sampling, and perturbation sensitivity associated with high-curvature training regimes such as Edge of Stability (EoS). We do not propose Neural ODEs, continuous-time models, Sharpness-Aware Minimization (SAM), or EoS metrics as default prescriptions. Instead, we propose a minimum reporting standard that makes robustness claims testable and supports escalation to more complex modelling or optimization only when stress tests justify the cost.

Background: Patient-facing AI intake chatbots can enter clinical use without regulatory review, so their errors may first surface with actual patients. Objective: To develop and pilot a remote, voice-based standardized patient (SP) method for evaluating an intake chatbot's history-taking, documentation, and patient-centeredness. Methods: Four primary care cases had Easy and Difficult variants. In Phase 1, five SPs completed 40 encounters with internal medicine residents rating portrayal fidelity. In Phase 2, SPs completed 80 encounters with a ChatGPT 4o chatbot, recorded whether critical elements were asked about and addressed satisfactorily, and rated summaries; four debriefing interviews were analyzed. In Phase 3, two blinded clinicians rated transcripts and summaries from 101 encounters (32 resident-led, 69 chatbot-led). Results: Portrayal fidelity averaged 4.52 of 5. The chatbot asked about 59.3% of critical elements in Easy versus 50.2% in Difficult variants (adjusted OR 1.49; 95% CI, 1.15 to 1.93) and addressed 73.3% versus 64.6% (OR 1.47; 1.12 to 1.93); near-ceiling summary quality did not differ. SPs reported missed information and rushed pacing. Reviewer agreement was moderate to substantial for element judgments (AC1 0.49 to 0.70) but poor for totals (ICC 0.10 and 0.21). In exploratory comparisons, residents asked about more elements (adjusted difference 12.5 percentage points; 95% CI, 7.0 to 18.1) and satisfactorily addressed more (9.2 points; 3.8 to 14.6); totals did not differ. Conclusions: SP portrayal was consistent enough for pre-clinical chatbot evaluation, and the method detected lower coverage in harder cases. Documentation ratings alone may miss conversational gaps, which require direct measurement.

Title and abstract screening is among the most labor-intensive steps of systematic reviews. Jev, a low-cost AI decision model, scores records against eligibility criteria without screening labels. We scored 169,982 records from 128 reviews and, after tuning on development reviews, evaluated 23 held-out reviews once and 31 Cochrane reviews post hoc. Jev outranked the best free ranker in all 23 held-out reviews (mean AUC 0.966 vs 0.831) and came within 0.011 AUC of Claude Opus 5.5 on 1999 records at about 1/700 of its list-price cost (US$0.02/1000 records). Combined with active learning, it raised work saved at 95% recall from 0.659 to 0.806. A fixed score threshold fell below 95% recall in 3 of 28 Cochrane reviews, whereas Jev ranking with statistical stopping (post hoc) reached 95% recall in all 51 reviews, reading 86-88% of records. Label-free ranking with Jev was accurate and inexpensive, but stopping safely required screening decisions.

BackgroundCochrane Risk of Bias 2 (RoB 2) assessment is methodologically demanding and resource intensive. Large language models may assist this process but can produce variable judgments. AimTo develop a rule-constrained LLM system for RoB 2 assessment and evaluate its temporal stability and agreement with human reviewers. MethodsRoB2-Evaluator was developed using GPT-5.2 with fixed RoB 2 guidance and a locked instruction framework. Seven RCTs were used for calibration before version freezing. The frozen system was evaluated on 10 separate RCTs, with repeat assessment after one week and comparison against consensus judgments from two independent human reviewers. ResultsTemporal agreement was 90.0% (weighted {kappa} = 0.93), and agreement with human consensus was also 90.0% (weighted {kappa} = 0.93). Methodological rationales were stable in 94.3% of repeated comparisons. Full AI-human conceptual alignment occurred in 85.7%. ConclusionRoB2-Evaluator showed encouraging post-freeze stability, human agreement, and rationale consistency, supporting larger independent evaluations.

Background Medicine shortages are monitored within national borders, but complementarity across national markets is poorly characterised. Objective To characterise medicines supplied through South Korea's national access programme and to quantify this complementarity between South Korea and Italy. Methods In this cross-sectional study, we linked shortage records from the Italian Medicines Agency and Korean Ministry of Food and Drug Safety, Korea Orphan and Essential Drug Center supply notices, and national reimbursement lists. The primary outcome was the proportion of shortage configurations with a reimbursed counterpart in the partner country, identified by normalised ingredient, pharmaceutical form, and route, and not recorded as withdrawn or in shortage. The primary analysis covered Jan 1 to June 30, 2026, and the secondary analysis covered 2023-26. Results Among 341 formulations supplied through the Korean national access programme, 242 (71.0%) lacked domestic marketing authorisation at first observed supply or were supplied after market withdrawal. Of 338 Italian shortage configurations in the first half of 2026, 228 (67.5%) had an available reimbursed Korean counterpart; over 2023-26, 332 of 549 (60.5%) did. Conversely, 27 of 47 Korean configurations (57.4%) had such an Italian counterpart; over 2023-26, 93 of 184 (50.5%) did. In both directions, ingredients on EU critical or Korean essential medicine lists more often had a cross-border match than unlisted ingredients. Conclusions About half of shortage or access-gap medicines had a potentially available reimbursed counterpart in the other country. Cross-national linkage of medicine data may help identify potential counterparts, although these require regulatory, clinical, and procurement review.

Large language models (LLMs) screen titles and abstracts without review-specific training, but generating screening decisions as text takes processing time and incurs API charges. We evaluated Jev, a non-generative model returning classification probabilities, on 4527 records from two systematic reviews of bipolar disorder treatments. We used human reviewers' decisions to retain records for full-text assessment as the reference standard. In the primary analysis, we asked Jev yes-or-no questions ("Noul") about whether to retain each record. Records were retained when their retention probability was at least 50%, a cutoff specified before the final evaluation runs. We also evaluated lower retention thresholds. For comparison, we gave Jev explicit include/exclude options ("Choice") and used the same prompts with GPT-5 mini and GPT-6 Astra; both LLMs used default reasoning. In the primary analysis at 50%, sensitivity and specificity were 99.6% and 98.9% for light therapy and 93.2% and 93.5% for adjunctive pharmacotherapy. Lower cutoffs selected on these data retained all reference-positive records. Compared with assessing every record manually, these cutoffs reduced the number requiring human assessment by 96.9% for light therapy and 61.3% for adjunctive pharmacotherapy. For light therapy and adjunctive pharmacotherapy, sensitivity was 92.3% and 90.0% with Jev using explicit options at 50%, 87.2% and 90.0% with GPT-5 mini, and 92.3% and 80.8% with GPT-6 Astra. Costs for screening all 4527 records once were US$0.20 for Jev using explicit options, US$2.59 for GPT-5 mini, and US$33.45 for GPT-6 Astra. Jev's probabilities could provide graded information for reviewers assessing titles and abstracts.

High attrition rates across clinical drug development impose immense clinical and financial burdens, yet prospectively forecasting whether an individual trial will achieve its primary endpoint remains challenging. Here we present a machine-learning system that predicts clinical trial success across phases I-IV, independently modeling studies with efficacy versus safety/toxicity primary endpoints. To prevent lookahead bias and outcome leakage, models are trained on registration-time features, including target and mechanism-of-action precedent, drug modality and novelty, trial design and operational characteristics, and frozen literature-derived target-choice embeddings. Each prediction yields a continuous Predictive Trial Index (PTI) between 0 and 1 reflecting success probability. On a test set of trials initiated in 2021 or later, the efficacy model achieved an area under the receiver operating characteristic curve (AUROC) of 0.828 (95% CI 0.812-0.845; n = 2,347) and the safety/toxicity model reached 0.809 (95% CI 0.787-0.831; n = 1,485). These models are embedded in inClinico Agent, an agentic AI system integrating frontier large language models (LLMs) with deterministic scoring engines, retrieval tools and specialized clinical skills. We also introduce ClinBench, a prospective benchmark of 2,040 ongoing trials evaluated by inClinico Agent and five frontier LLMs, establishing a leakage-free foundation for tracking prospective predictive accuracy as trials reach completion.

Background. Regulatory pressure to avoid confusingly similar drug names appears to have pushed manufacturers to coin more distinctive proprietary names, but no one has quantitatively analyzed this trend. Objective. To determine whether proprietary drug names became more orthographically distinctive (unusual in spelling) over time. Methods. We examined 1,321 US Food and Drug Administration (FDA) new molecular entity (NME) names (1985-2024), 2,769 FDA Orange Book trade names (1982-2026), 177,337 US pharmaceutical trademarks (1980-2026), and 1,273 generic names (1982-2025). Compared to prior names (including those already marketed) and ordinary English, we scored each name's character trigram (three-letter sequence) surprisal (statistical unexpectedness) and its edit-distance similarity to the closest existing names. Linear regression tested time trends, controlling for name length. Results. Mean NME trigram surprisal rose in every decade between the 1980s and 2020s, from 7.21 to 8.50 nats (natural-logarithm units, higher meaning less predictable), an 18% increase (b = 0.038, p < .001, R2 = 0.34). Half of 2020s names (7% in the 1980s) contained a letter sequence found in no common English word (e.g., Jesduvroq, Omvoh). Nearest-neighbor similarity fell after the early 1990s. Trademark data showed the same trend (b = 0.016, p < .001). Generic names showed a weaker trend (b = 0.011, R2 = 0.063). Conclusions. Over four decades, drug names became more orthographically distinctive, consistent with namespace crowding, deliberate efforts to avoid confusing names, and branding strategy. Because unfamiliar letter sequences can introduce other error pathways, increased distinctiveness does not necessarily imply improved safety.

Children living with HIV face significant challenges with adherence to daily oral antiretroviral (ARV) treatment due to poor palatability, complicated dosing, and caregiver burden. Long-acting delivery systems designed specifically for children may help overcome these barriers. Microarray patches, also known as microneedle patches, are a needle-free drug delivery technology that could simplify treatment administration and improve adherence. This study explored the acceptability and usability of a pediatric ARV microarray patch prototype among stakeholders in South Africa and Uganda, two countries with a high burden of pediatric HIV. A formative study was conducted in 2024 with 48 participants, including caregivers, community health workers, health care providers, and decision-makers. Data were collected through qualitative methods: interviews, simulated use sessions, and observations. Participants demonstrated high usability of the prototype, with 97% completing all application steps and nearly all reporting the prototype as easy to use and expressing confidence in successful administration. Most participants found the prototypes size, wear time, and temporary skin changes acceptable, and all preferred a weekly patch over daily oral treatment. Applying multiple patches at once to achieve a therapeutic dose was considered feasible. Decision-makers foresaw strong programmatic fit of a pediatric ARV microarray patch if caregivers apply the patch at home following initial facility-based guidance. Overall, the pediatric ARV microarray patch prototype was perceived as highly acceptable and usable across diverse end users, findings that generally align with studies of microarray patches for other use cases (e.g., contraception, HIV pre-exposure prophylaxis, vaccine delivery). The simplicity, comfort, and convenience of a pediatric ARV microarray patch could substantially improve treatment adherence and quality of life for children living with HIV. Further development and clinical evaluation are warranted to advance this promising technology toward real-world use.

Clinicians routinely make decisions for patients whose comorbidities, prior therapies, laboratory abnormalities, or demographic characteristics are not reflected in the supporting evidence, even though these factors can influence treatment selection. Given this gap in the evidence base, we sought to evaluate how AI models respond to clinical questions in which patient characteristics or the treatment comparison and outcome of interest may affect the applicability of available evidence. We developed and publicly released Precision Evidence Bench, a benchmark of 209 synthetic clinical questions with specified population, intervention, comparator, and outcome (PICO) elements. Each answer is assessed using a five item rubric that evaluates whether a single cited source matches the PICO and whether the answer faithfully reports its sources. The item verdicts determine one of five ratings, ranging from Precision match to Unsupported claim, without requiring a reference answer. We evaluated four frontier large language models (LLMs) using web search across all 209 questions, then repeated the evaluation with additional access to a corpus of precision evidence-based findings (pEBFs) designed to address these evidence gaps, using two corpus sizes. For every model, Precision match rates increased with the amount of precision evidence available, from 13 to 18% with web search alone to 17 to 28% with 100 million pEBFs and 41 to 56% with 500 million pEBFs. These findings suggest that access to matching evidence is an important determinant of LLM performance on precision medicine questions.

Background. Large language models (LLMs) are increasingly used to support systematic review and meta-analysis, but complex evidence flow, trial-family identification, study-level data extraction, and statistical execution remain vulnerable to omissions, untraceable data paths, and scientifically flawed steps. It remains unclear whether gains are driven by modular skills or by fixed orchestration, and whether these effects differ across base models. Methods. We conducted a controlled, multi-model benchmark in which six LLMs completed the same randomized controlled trial (RCT) meta-analysis task under three conditions: (A) no meta-analysis-specific skills; (B) autonomous access to modular meta-analysis skills; and (C) the same skills executed under a Quality-First fixed directed acyclic graph (DAG). Each model was run three times per condition (54 runs total). The primary outcome was the scientific performance score (SPS, 0-100), rated by two meta-analysis experts; one expert was masked to explicit model and condition labels during SPS scoring, and final scores were the arithmetic mean of the two raters. Inter-rater reliability was high (SPS ICC(A,2) = 0.991). Results. Mean SPS was 43.50 (condition A), 55.65 (B), and 54.44 (C). Relative to unassisted models, modular skills improved mean SPS by +12.15 points (unadjusted 95% CI 3.15-21.16; Holm-adjusted P = 0.033). The combined modular-skills-plus-fixed-DAG condition showed a positive but not statistically significant difference relative to unassisted models (C-A = +10.94; Holm-adjusted P = 0.073) and no incremental gain over skills alone (C-B = -1.21; Holm-adjusted P = 0.746). No condition-by-model interaction was detected (P = 0.801). Conclusions. In this benchmark task, access to modular skills improved mean scientific performance relative to unassisted execution; fixed orchestration of the same skills did not provide a detectable incremental benefit. The findings are task-specific and do not establish general effects for other meta-analysis settings, models, or orchestration designs. Keywords: Large language models; meta-analysis; Agent skills; Workflow orchestration; Evidence synthesis

A fundamental challenge in drug development is the frequent failure of early laboratory research to translate into clinical benefit. One promising solution is to confirm findings from exploratory single-laboratory studies across multiple laboratories before clinical testing. We investigated this approach following the conduct of preclinical multi-laboratory studies across different fields of medicine. For this, we evaluated effect sizes, experimental rigor, and a set of criteria to identify determinants of confirmation success. When tested under increased rigor, only a fraction of multi-laboratory studies confirmed the initial results. The underlying effect size reduction was associated with outcome-relevant experimental differences between exploratory and confirmatory stages. In summary, multi-laboratory studies proved highly informative and served as an effective filter for promising treatments.

Importance. Observational studies dominate evidence in cardiovascular outcomes research, yet no widely adopted reporting standard requires authors to disclose how much of an observed association is attributable to systematic error. The negative-control calibration framework of the Observational Health Data Sciences and Informatics consortium returns a binary answer: a calibrated P-value either crosses 0.05 or it does not. That answer conflates "is there a signal" with "is the signal credible." Objective. To define the Bias Attribution Fraction (BAF), a continuous estimator in [0, 1] that quantifies the fraction of an observed log relative risk that is attributable to systematic bias, together with its 95% credible interval, and to evaluate whether reporting calibrated P-value plus BAF plus interval width produces more defensible decisions than reporting the calibrated P-value alone. Design. Three-part study: (i) metric definition and interval construction; (ii) Monte Carlo simulation across 1920 conditions by 1000 repetitions = 1,920,000 simulated target estimates (a primary grid plus a sign-flipped companion grid of identical size, pooled for all analyses), with truth known by construction; (iii) two-arm beta-blocker target trial emulation in MIMIC IV, adhering to the TARGET guideline, with targeted maximum likelihood estimation (TMLE) and 12 negative-control outcomes. Setting. MIMIC IV (critical care) and MIMIC IV ECG, 2008 to 2019. Participants. Adults with a first intensive care unit stay and a recorded 12-lead ECG. Exposures. Two clinical questions are evaluated: (1) reaching at least 50% of guideline-recommended target dose of an index beta-blocker within 7 days of admission, sustained to 12 months, versus lower or no exposure; (2) receiving guideline-directed medical therapy (GDMT) with at least two of three foundational drug classes at adequate dose for 12 months versus fewer. Main Outcomes and Measures. One-year all-cause mortality. Primary analytic outputs: (i) the calibrated P-value from a negative-control empirical null distribution with 12 outcomes; (ii) BAF-hat with a Markov chain Monte Carlo 95% credible interval; (iii) a verdict drawn from a truth-calibrated lookup table mapping the BAF-hat by interval-width cell to a probability of being bias-dominated. Results. Across 1,920,000 simulated repetitions, the calibrated P-value alone yielded a misuse rate (declared usable while truth was bias-dominated) of 36.02%. Adding the BAF point estimate reduced misuse to 16.48%, and requiring the whole credible interval of BAF to lie below 0.5, clear of the bias-dominated zone, reduced misuse to 10.16%. Requiring in addition a calibrated probability of bias dominance below 15% reduced misuse to 3.02% at the cost of declaring 22.1% of estimates usable (versus 61.8% for the calibrated P-value alone). Read from the bias-dominated view, meaning that withholding a repetition is a positive call and being truly bias-dominated is a positive truth, the BAF estimator achieved an area under the receiver operating characteristic curve of 0.922, compared with 0.787 for the calibrated P-value. In MIMIC IV, question 1 produced a calibrated P-value of 0.894 and BAF-hat = 0.875 (95% credible interval 0.385 to 0.992); the apparent one-year mortality reduction of 16.9% was entirely attributable to systematic bias and the association was not usable as effect evidence. Question 2 also failed layer 1, with a calibrated P-value of 0.1026, and BAF-hat = 0.460 placed it in the "competitive, no verdict" zone. Conclusions and Relevance. Pairing the calibrated P-value with BAF and its 95% credible interval turns a binary credibility judgement into a three-channel report (signal existence, effect size, and bias fraction). Despite TARGET-adherent design, doubly robust TMLE, and 12 negative controls, both clinical questions in this study still failed to qualify as effect evidence, with question 1 demonstrating complete bias domination and BAF quantifying the systematic error that remained. The reporting standard proposed here can be applied to any observational study that includes negative controls, without requiring additional data.

Systematic reviews condense clinical trials into evidence tables, yet clinicians can interrogate these tables only through database queries, and many questions concern attributes that the table does not record, such as a drug's target class or a harmonised endpoint. Here we introduce FD-SCoPE, a language-model framework that answers both kinds of question, exposes the query, the selected trials and the derivation rule behind every answer, and learns from expert corrections. On an oncology evidence table of 159 immune checkpoint inhibitor trial records, FD-SCoPE completed all 140 clinician-style tasks (alternatives, 90.7--97.9%). For questions needing derived attributes it retrieved 99.3% of relevant trial records at a positive predictive value of 89.8% and outperformed four alternative approaches (derived-value F1 77.7% versus 64.8--73.4%). Corrections on 299 questions, simulated from reference answers, raised F1 on 1,201 unseen questions from 77.9% to 84.9%. Language models coupled with executable queries, verified programs and expert feedback can give clinicians auditable access to trial evidence.

Background Large language models (LLMs) show promise in scientific literature reviews, but cloud-based proprietary models like GPT-4 raise concerns around privacy, copyright, and licensed article access. Using a scoping review on AI applications for treatment effect estimation as a use case, this study evaluated local open-source LLMs for automated article screening in scientific literature reviews and developed a practical workflow for their implementation. Methods We extracted a sample dataset of 300 records from an ongoing scoping review on machine learning methods for treatment effect estimation using real-world medical data. We divided the dataset into training (n = 200) and testing (n = 100) datasets. We evaluated 11 locally deployed open-source LLMs from five model families (GPT-OSS, Gemma, Qwen, Mistral, Llama) for both abstract screening and full-text screening. Full-text screening was done using text converted by two PDF-to-text conversion tools: Docling and PyMuPDF4LLM. Performance was assessed using raw accuracy with 95% Wilson's confidence interval, relaxed accuracy, Cohen's kappa, and runtime. During workflow development, prompt engineering, output extraction strategies, and generation settings were iteratively optimized using the training dataset. Results GPT-OSS-20B achieved the highest raw accuracy for title and abstract screening (0.890 on the training set and 0.850 on the test set), followed closely by Gemma-4-31B-it (0.875 on the training set and 0.830 on the test set) and Gemma-4-26B-A4B-it (0.810 on the training set and 0.790 on the test set). In full-text screening, slightly higher raw accuracy and reduced false-positive rates were observed, with GPT-OSS-20B and Gemma-4 variants consistently demonstrating strong performance. Only minor variations were found between the two PDF-to-text conversion methods evaluated, though runtime varied substantially across models. Based on these findings, we compiled a set of practical recommendations for local LLM-assisted article screening. Conclusions Local LLMs can effectively support literature screening when integrated into a carefully designed workflow. Recent generations of open-source local models achieved performance comparable to human reviewers and proprietary LLMs. Local deployment also provides advantages in data privacy, copyright compliance, customization, and cost-efficiency. While human oversight remains essential, the proposed workflow demonstrated that local LLMs represent a practical and reproducible approach for accelerating evidence synthesis.

Aims. It is unknown whether large language models (LLMs) can predict a trial's result at the design stage, from the information available when it is registered. We tested three LLMs prospectively on the Late-Breaking Science programme of the European Society of Cardiology (ESC) Congress 2026. Methods and results. Of 139 trials, 50 were selected by pre-specified criteria, and predictions were locked and registered on the Open Science Framework before the congress. Claude Fable 5, GPT-5.5 and Gemini 3.5 Flash, without web access, received each title and an investigator-compiled summary of the registered design. They gave the probability that the primary endpoint would be met, and a point estimate with an 80% prediction interval for a locked effect measure. Results were adjudicated from publications (19) or presenters' slides (31). Of 48 scorable trials, 28 (58%) were positive. The area under the receiver operating characteristic curve was 0.80 (95% CI 0.66-0.92) for Claude, 0.79 (0.66-0.91) for GPT and 0.74 (0.59-0.87) for Gemini, with accuracies of 73%, 71% and 67%. Among 26 trials whose effect was reported exactly as locked, the direction was correct in 88% of predictions and the median multiplicative error of ratio estimates was about 15%. The 80% prediction intervals contained the observed effect in 73.1% (Claude), 84.6% (GPT) and 57.7% (Gemini). Conclusion. From the registered design alone, LLMs showed statistically significant discrimination between positive and negative trials and usually predicted the direction and approximate size of the effect. Such predictions might help trial design, and further studies are needed.